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ABSTRACT 

We investigate the possibility of observing the exotic decay mode of the top quark into the hghtest 
stop {ti) and neutrahno (xj*) in the minimal supersymmetric standard model with R-parity at the 
upgraded Tevatron. First we determine the allowed range for the branching fraction B(t iiXi) 
in the region of parameter space allowed by the i?t, data and the CDF 6677 + I^t event, and then 
consider all possible backgrounds and investigate the possibility of observing this final state at the 
Tevatron. We find that this final state is unobservable at Run 1. However, Run 2 can provide 
significant information on this new decay mode of the top quark: either discover it, or establish a 
strong constraint on the masses of ti and x\ given approximately by M-j^o > Mj^ — 6 GeV. 
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1. Introduction 



Because of its large mass, the top quark has the potential to be a sensitive probe for new 
physics. In strongly interacting theories, such as top condensation and extended technicolor, 
the top quark plays an essential role in the electroweak symmetry breaking and in the 
understanding of flavor physics. In weakly interacting theories, such as supersymmetry 
(SUSY) [1], the heavy top quark provides a solution to the electroweak symmetry breaking 
and makes it possible that the top quark may decay into its lightest superpartner i (ti) plus 
the lightest neutralino (xi)- Assuming that ti decays dominantly into ex?, this SUSY decay 
mode of the top quark will give rise to a new final state in ti production at the Fermilab 
Tevatron, ti WbcXiXi- 

A careful study of this final state is well motivated since the presently allowed parameter 
space [3] [4] of the minimal supersymmetric standard model (MSSM), implied by the 6677 + 

event at CDF [5] and the LEP Rb data , will allow the top quark to have this new decay 
mode t iiXi- is also implied in this scenario that Mf^ < M-± [3,4], where xf is the 
lightest chargino, so the decay ti bxi is not allowed. In the case that the ti is lighter 
than the next-to-lightest neutralino the dominant decay of ti is ti cx\ via 

one loop processes [6], with a branching fraction of almost 100% i. Therefore searching for 
this final state may be a powerful tool for probing SUSY at FNAL. 

The possibility for detecting the WhcXiXi filial state in ti production was first discussed 
in Ref. [7] where the focus was mainly on the background ti W~W^hh. In this article, 
in the framework of the MSSM with the lightest neutralino being the LSP, we will present a 
detailed analysis including all the possible backgrounds. In particular, we first determine the 
allowed range for Bit tiXi) in the region of parameter space allowed by the Rb data and 
the ee77 + I^t event at CDF, and then show what additional constraints can be imposed 
on the allowed parameter space if this final state is not observed at the Tevatron. We find a 
lower bound of 0.07 for B(t ^iXi) in the presently allowed parameter space [3] [4], and, as 
a result, we find that Run 2 can either discover this new decay mode or provide an additional 
strong constraint given approximately by M^o > M^^ — 6 GeV. However, our results show 
that even if B(t ^iXi) is as large as 0.5, it is unobservable at Run 1. 

This paper is organized as follows. In Sec. 2 we examine the range of values for B{t 
tiXi) in the region of parameter space allowed by the Rb data and the 6677 -|- I^t event. 

^Electroweak baryogenesis in SUSY requires a light stop to have a strong first order phase transition [2]. 
•^The four-body decay mode ti bxifij-i is kinematically suppressed by both xf — and W^— propagators 
and thus its partial width is negligibly small. 



1 



In Sec. 3 we examine all possible backgrounds and investigate the possibility of observing 
tt —>■ WbcXiXi the Tevatron. And finally in Sec. 4 we present a summary. 



2. Bounds for B{t tiX?) 



It was found that in order to explain all of the presently available low energy data the 
lightest mass eigenstate (ti) of the stop squarks is likely to be the right-stop (tji) with mass 
of the order of Mw [4]. So we assume ti = in our analyses. The interaction Lagrangian 
of top (t), stop(ti) and neutralino (x5) is given by [8] 

Atix? = -V2t(APi + BPr)x% + H.C., (1) 

where Pl,r = (1 ^ 75)/2 and 

2 2 (7 9^ 

A = e{Ni,Cw + N,2Sw) + -^-T^{N^2Cw-NuSw), (2) 

B = — ' ^ ■ 3 
2Mw.sin/3 ^ ^ 

Here Sw = sin 9 w,Cw = cos 9w, and Nij are the elements of the 4x4 matrix N which 
diagonalizes the neutrahno mass matrix [1]. The decay t — > iiXi been calculated to 
one-loop level in Refs. [9] and [10]. Here we neglect the loop corrections, which are only on 
the order of 10%; the partial width is given at tree level by 

nt^hx',) - Y^AV2(^,^M|o,M|)[(|.lp + |S^(M,^ + M|o-M^) 

+ARe{A*B)MtM^o\ , (4) 

where X{x, y,z) — {x — y — z)^ — Ayz. 

The parameters involved in r(t — > iixT) 

M^^,M2,Mi,At,tan/3, (5) 

where M2 and Mi are gaugino masses corresponding to SU{2) and is the coefficient 

of the H1H2 mixing term in the superpotential, and tan (3 = V2/V1 is the ratio of the vacuum 
expectation values of the two Higgs doublets. If the 6677 + I^t event is due to pair 
production (depending on the scenario, the selectrons decay into either or exi, followed 
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by X2 ~^ Xil oi' Xi ~^ G'y, respectively, where G is the gravitino), the region of the parameter 
space allowed by both the 6677 + I^t event and the Rb data are given as [3] 

50 < Ml < 92 GeV, 50 < M2 < 105 GeV, 
0.75 < M2/M1 < 1.6, -65 < /i < -35 GeV, 
0.5 < |/i|/Mi < 0.95, l<tan/3<3, 

33 < M^o < 55 GeV, 45 < M^^ < 80 GeV. (6) 

If the 6677 + ¥t event is due to cr pair production, the allowed region is [3] 

60 < Ml < 85 GeV, 40 < M2 < 85 GeV, 

0.6 < M2/M1 < 1.15, -60< fi< -35 GeV, 

0.5 < |/i|/Mi < 0.8, l<tan/?<2.2, 

32 < M^o < 50 GeV, 45 < M^^ < 80 GeV. (7) 
In the region of Eq. (^) we obtain 

0.07 < B{t tix?) < 0.50, (8) 



and in the region of Eq. , 



0.10 < B{t tix?) < 0.50. (9) 



So, if the 6677 + event is explained by the MSSM with the lightest neutralino being the 
LSP, the exotic decay t — * tiXi niust occur at a branching ratio larger than 0.07. 

Upper bounds for this exotic decay of the top quark can also be derived from the available 
data at FNAL. Currently, the FNAL top quark pair production counting rate is interpreted 
as a measurement of a{ti) x B'^{t bW). Since the final states ti WbcXiXi ^i-nd ti — > 
cXiXiCXiXi do not have enough leptons or jets to be included in the dileptonic, leptonic or 
hadronic event samples, they are invisible to the current counting experiments at FNAL. So 
the quantity [1 — B{t tiXi)]'^, which gives the fraction of events in which both the t and 
i decay normally 0, should lie within the measured range of cr[tt|exp/cr[i^^QCD- Note that in 
our analyses we neglected the SUSY effects [13] [14] in ti production and thus the theoretical 
value of a[ti\ is given by the SM value a[tt\QCD- The production cross section measured by 



"^Here we assume that the only exotic decay mode of top quark in R-parity conserving MSSM is i — > iiXi- 
If charged Higgs is light enough, t H^b is also possible; its phenomenological implications at Tevatron 
have been studied [11]. The FCNC decays t cZ,c^,cg,ch are negligibly small in R-parity conserving 
MSSM [12]. 
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CDF with an integrated luminosity of 110 pb^-*^ is o-[tt\cxp = 8-5^3;4, G.Stf^, 10.7t4;4pb in the 
dilepton, lepton+jets and all-hadronic channels, respectively [15]. The SM expectation for 
top mass of 175 GeV is ^[ttlQCD = 5.5io;4 pb [16]. By comparing (T[tt]cxp from each channel 
with (T[tt]QCD[l — B(t ^iXi)]^) we find that the 2a upper bounds on B{t —>■ iiXi) for the 
various channels are given by 



Here the upper bound from lepton+jets channel is comparable to the upper bound of 0.25 [7] 
obtained by a global fit to the available data. If the possible enhancement of tt production 
cross section from gluino pair production is taken into account, the upper bound for B{t ^ 
iiXi) can be relaxed to 0.5 [14]. 



Under the assumption that the top (or anti-top) decays via the normal weak interactions 
to Wb, the anti-top (top) decays to tiXi, ^^e light stop decays to cXi, then the final 
state of interest is WbcXiXv Due to the large QCD backgrounds, it is very difficult to search 
for the signal from the hadronic decays of W at the Tevatron. We therefore look for events 
with the leptonic decay of the W. Thus, the signature of this process is an energetic charged 
lepton, one 6-quark jet, one light c-quark jet. plus missing Et from the neutrino and the 
unobservable Xi's- We assumed silicon vertex tagging of the 6-quark jet with 50% efficiency 
and the probability of 0.4% for a light quark jet to be mis-identified as a 6-jet. The potential 
SM backgrounds are: 



(1) 6g(g)-.tg'(g0; 

(2) qq' -^W* ^ tb; 

(3) Wbb; 

(4) Wjj; 

(5) ti W-Wnl; 

(6) gb^tW; 

(7) qg ^ q'fb. 




0.44 dilepton channel 
0.23 lepton -f- jets channel 
0.41 all — hadronic channel. 



(10) 



3. Observing tt — ^ WbcxiXi ^he Tevatron 
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The quark-gluon process (7) can occur with a W-boson intermediate state in either the t- 
channel or the s-channel. We found backgrounds (6) and (7) to be neghgible since they have 
an extra jet, and can mimic our signal (before 6-tagging) only if a jet is missed in the detector. 
The background process (5) can mimic our signal if both 1^'s decay leptonically and one 
charged lepton is not detected, which we assumed to occur if the lepton pseudo-rapidity and 
transverse momentum satisfy ri{l) > 3 and pri^) < 10 GeV, respectively. 

To simulate the detector acceptance, we made a series of basic cuts on the transverse 
momentum {pr), the pseudo-rapidity {r]), and the separation in the azimuthal angle-pseudo- 
rapidity plane ( AR = ^ (A^)^ -|- (Ar;)^ ) between a jet and a lepton or between two jets. 
These cuts are chosen to be 

Pt, Pt\ Pt'"" > 20 GeV , (11) 
Vict, m < 2.5 , (12) 
ARjj, ARji > 0.5 . (13) 

Further simulation of detector effects is made by assuming a Gaussian smearing of the energy 
of the final state particles, given by: 

AE/E = 30%/V^e 1%, for leptons , (14) 
= 80%/v^® 5%, for hadrons , (15) 

where © indicates that the energy dependent and independent terms are added in quadrature 
and E is in GeV. 

In order to substantially reduce the background, we apply a cut on the transverse mass 
defined by = \/ (Pj- + P^^^^^y — (Pj. + P^^^^y. Without smearing, rriT is always less than 
Mw (and peaks just below Mw) if the only missing energy comes from a neutrino from W 
decay, which is the case for most of the background events (single top, Wbb, Wjj). Smearing 
pushes some of this above Mw- For the signal mr is spread about equally above and below 
Mw, due to the extra missing energy of the neutralinos. Therefore we also require 

rriT > QOGeV. (16) 

The results with different cuts are shown in Table 1. For convenience the numerical 
results shown are obtained without including the appropriate branching ratios; the actual 
cross sections are found by multiplying the given values by the branching fraction factors 
X — B{t — > cXiXi) ^^d 1 — X — B{t — > bW). The products of the appropriate branching 
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fractions in each case are given in the last column of Table 1. In our numerical evaluation, 
we assumed Mt = 175 GeV, y/s = 1.8 TeV and an integrated luminosity of 0.1 fb~^ for Run 
1, and ^/s = 2 TeV and an integrated luminosity of 10 fb~^ for Run 2. 

At Run 1, with the basic and mrp cuts the number of background events is always less than 
1, and the number of signal events is always less than 9 for any value of x = B{t ^ iiXi) 
allowed by Eq.(H) or @. Thus the signal is unobservable at Run 1 under the criteria 
S > + S, which corresponds to the 95% confidence level (CL.) if we assume Poisson 
statistics. The cut hurts the signal, but, as we pointed out above, it reduces the back- 
ground much more than the signal. Even when the rriT cut is relaxed, this final state is 
unobservable at Run 1. 

At Run 2 this signal is observable even for quite small values of B{t tiXi)- -^^S- ^ 
we show B{t iiXi) versus M^o for the signal to be observable at 95% CL. The region 
above each curve is the corresponding observable region. From this figure we can see that 
for B{t —>■ iiXi) > 0-0''' (^^^ minimum value allowed by Eqs. 8 and 9), Run 2 can detect this 
signal at 95% CL. if (approximately) M^o < M^^ — 5 GeV. This means that if this signal is 
not observed at Run 2, an additional constraint, given approximately by 

M^o > - 5 GeV, (17) 

can be placed on the region of Eq.(|^) allowed by the Rb data and the 6677 + I^t event at 
95% CL. 

In Fig. 2 we present B{t ^iXi) versus M^o for the signal to be observable under the 
stricter discovery criteria S > 5^/B. In this case we see that if this signal is not observed at 
Run 2 the additional constraint of approximately 

M^o > - 6 GeV, (18) 

can be placed on the allowed region of Eq. (H) . 

Note that a lower limit of 67 GeV for ti has been obtained from the direct search for 
ti cXi under the restriction M^^ — M^o > 10 GeV [17]. Our results in Fig. 2 show that 
under the condition M^^ — M^o > 6 GeV, Run 2 can explore the entire presently allowed 
parameter space. In order words, if this final state is not seen at Run 2, only a very small part 
of parameter space will be allowed, which satisfies Eq.(|D or Eq.(|^) plus M^o > Mj^ —6 GeV. 

We conclude by noting that the more precise tt cross section measured at Run 2 will 
further strengthen the upper bound for B{t ^iXi) given in Eq.([ToD. However, as our 
results show, is it also possible to discover the decay t iiXi ^VLn 2 for values of 
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B(t iiXi) much smaller than the upper limits obtained by measuring the tt cross section. 
In any event, further constraints on the parameter space can be made through searching for 
this final state. 

4. Summary 

In the framework of the MSSM with the lightest neutralino being the LSP, we first 
determined the allowed range for the branching fraction B(t ^ ^iX?) in the present allowed 
region of parameter space and found a lower bound of 0.07. Then we investigated the 
possibility of observing t — > tix^ at the Tevatron by searching for the final state tt 
WbcXiXi- We found that: 

(a) This final state is unobservable at Run 1 ; 

(b) Run 2 can either discover this new decay mode or place the additional constraint 
M^o > — 5 GeV if S* > 3V B + S is required for discovery of the signal, or 
M^o > GeV if we require S > by/B. 

In our analysis, we neglected the possibility of the enhancement of the top pair production 
cross section in the MSSM. In particular, the gluino pair production might be significant, [14] 
and would give rise to a new final state tttt*. This will not affect our conclusion significantly 
since it will give a final state with more jets than the signal we are considering. With such 
a mechanism of exotic top pair production, the upper bound on B{t — > iiXi) can be relaxed 
up to 50% [14], which will enhance the observability of this new mode at the Tevatron and 
strengthen our conclusion. 
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Figure Captions 

Fig. 1 The value oi B{t iiXi) versus M^o for the signal to be observable at Run 2 
under the criterion S > 3-\/S~+^. The region above the curve is the observable region. 
Fig. 2 Same as Fig. 1, but under the criterion S > 5y/B. 
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Table 1: 

Typical signal and background cross sections in units of fb after various cuts at the Tevatron. 
The basic cuts are p^^ > 20 GeV , 77aii < 2.5 and AR > 0.5, and the transverse mass cut is 
rriT > 90 GcV. The signal tt —>■ WbcXiXi results were calculated by assuming Mi^ = 60 GeV 
and M^o = 40 GeV. We have also everywhere assumed the use of silicon vertex tagging of 
the 6-quark jet with 50% efficiency and the probability of 0.4% for a light quark jet to be 
mis-identified as a 6-jet. The charge conjugate channels have been included. The numerical 
results do not include the branching fractions for the top and anti-top decays; the actual 
cross sections are found by multiplying the given cross sections by the branching fraction 
factor in the last column, where x stands for B{t ^ ^iXi)- 





Run 1 


Run 2 


BF factor 




basic cuts 


basic-|-mT cut 


basic cuts 


basic-|-mT cut 


tt ^ W^6cx?x? 


206 


114 


287 


162 


2x(l-x) 


qh ^ q'f 


79.5 


2.31 


11(3 


1.96 


1 - ,r 


qq' —>■ tb 


32.0 


1.77 


39.0 


2.25 


1 — X 


Wbb 


113 


2.04 


132 


2.50 


1 


Wjj 


392 


2.30 


505 


2.88 


1 


tt 


5.69 


2.72 


7.9 


3.82 


{l^xf 
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